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SIMULATION  OUTPUT  ANALYSIS  FOR 
GENERAL  STATE  SPACE  MARKOV  CHAINS 

by 

Peter  W.  Glynn  and  Donald  L.  Iglehart 
Stanford  University 

1.  Introduction 

The  statistical  analysis  of  simulation  output  has  been  the  primary  focus 
of  recent  research  in  simulation  methodology.  Methods  have  been  developed 
which  permit  the  simulator  to  construct  confidence  intervals  for  steady-state 
characteristics  of  the  system  being  simulated.  The  principal  methods  in 
current  use  are  autoregressive  modeling,  batch  means,  regenerative,  and 
replication.  With  the  exception  of  the  replication  method,  all  methods  are 
based  on  just  one  simulation  run.  These  methods  for  constructing  confidence 
intervals  are  all  based  on  central  limit  theorems  for  the  underlying  stochastic 
processes  being  simulated.  Thus  all  methods  are  only  valid  asymptotically 
for  long  simulation  runs. 

In  this  paper  we  discuss  three  new  methods  for  analyzing  simulation 
output.  All  three  methods  are  aimed  at  analyzing  the  simulation  output  of 
general  state  space  Markov  chains.  This  class  of  processes  encompasses  the 
embedded  jump  chain  generated  by  a  generalized  semi-Markov  process  (GSMP) . 
GSMP's  are  important  for  simulation  since  they  may  be  used  to  model  a  general 
discrete  event  simulation.  GSMP's  have  been  discussed  in  recent  papers  oy 
FOSSETT  (1979),  HORDIJK  and  SCHASSBERGER  (1981),  and  WHITT  (1980), 

The  first  method,  called  the  extended  regenerative  method  (ERM) ,  is  based 
on  some  recent  work  on  general  state  space  Markov  chains  by  ATHREYA  and  NEY 
(1978)  and  NUMMELIN  (1978).  This  method  involves  a  construction  which  creates 


regeneration  points  for  Markov  chains  which  do  not  hit  a  single  point  infinitely 
often.  While  this  idea  is  very  attractive  in  principle,  there  are  a  number 
of  practical  considerations  which  limit  its  application.  However,  the 
method  can  be  used  to  increase  the  rate  of  regeneration  points  when  using 
the  standard  regenerative  method.  For  more  details  on  this  method  and  some 
related  results  see  GLYNN  (1981). 

The  second  method ,  called  the  random  blocking  method  (RBM) ,  is  based  on 
blocks  of  the  process  which  begin  when  the  process  enters  a  given  set  in 
the  state  space.  This  method  is  reminiscent  of  the  regenerative  method  except 
the  blocks  created  here  are  not  independent  and  identically  distributed. 

Details  on  this  second  method  can  be  found  in  a  forthcoming  paper  by  GLYNN 
(1981). 

The  last  method  is  a  variation  of  the  method  of  autoregressive  modeling. 
This  method,  the  multivariate  autoregressive  method  (HARM),  fits  a  multi¬ 
variate  autoregressive  model  to  the  simulation  output  data.  The  model  fitting 
is  done  automatically  based  on  Akaike's  AlC-criterion;  see  AKAIKE  (1976)  for 
a  full  discussion  of  these  criterion.  A  forthcoming  paper  by  JOW  (1981)  will 
develop  this  method  and  other  related  methods  for  simulation  applications. 

This  paper  is  organized  as  follows.  Section  2  is  denoted  to  a  discussion 
of  GSMP's  and  their  relation  to  simulation.  The  ERM  is  covered  in  Section  3 


and  RBM  in  Section  4. 
illustrative  example 


2 


2.  Generalized  Semi-Markov  Processes 

In  a  discrete-event  simulation  a  finite  number  of  events  occurring  at 

random  times  cause  changes  in  the  state  of  the  system  being  simulated.  The 

number  of  events  active  at  any  given  time  is  a  function  of  the  state  of  the 

system.  This  type  of  simulation  is  well  modeled  by  the  generalized  semi- 

Markov  process  (GSMP)  which  we  now  describe. 

Let  S  be  the  finite  (or  countable)  set  of  states  which  describes  the 

GSMP  at  the  successive  transition  epochs  and  G  the  finite  number  of  events 

which  can  cause  a  transition.  If  G  “  ie, ,  e«,  ...»  e  },  then  we  let 

1  z  m 

G(s)  =  {e,  (s),  ...»  e  (s)}  denote  the  subset  of  events  active  when  the  GSMP 
1  n 

s 

is  in  state  s.  For  each  event  active  in  state  s,  associate  a  clock  which 
records  the  time  until  that  event  would  trigger  a  state  change.  If  in 
state  s,  the  clock  (associated  with  an  event  in  G(s))  with  the  minimal 
reading  triggers  the  next  state  change  when  it  t^ns  down  to  zero.  Let  C(s) 
denote  the  possible  clock  readings  when  the  GSMP  is  in  state  s: 

C(s)  =  {c  ^  :  c^  >  0  iff  e^  €  G(s);  c^  /  c^  for  i  j}  ^ 

where  IR™  is  the  Cartesian  product  of  m  copies  of  [0,<»).  Next  define  the 
space  Z  by 


Z  =  U  ({s}  X  C(s)) 
s  €  S 


and  the  process  (S,C)  =  { (S  ,  C  )  :  n  >  0}  which  lives  on  Z  and  represents 

^  n  n  “■ 

the  state  values  and  clock  readings  at  the  successive  transition  epochs.  The 
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process  (^>^)  is  a  general  state  space  Markov  chain  whose  transition  kernel 
will  be  defined  in  Section  3.  Finally,  the  GSMP  is  a  piece-wise  constant 
process,  {X  :  t  ^  0},  constructed  from  the  embedded  jump  process  (S,C)  in 
the  usual  manner. 


3.  The  Extended  Regenerative  Method  for  GSSMC's 

We  start  by  formalizing  the  notion  of  a  general  state  space  Markov  chain 
(GSSMC).  Let  E  be  a  complete,  separable  metric  space  with  E  its  associated 
Borel  field.  A  function  P  :  E  x  E  -*■  [0,1]  is  called  a  probability  transition 
kernel  if : 

i)  P(x,')  is  a  probability  on  (E,E)  for  each  x  in  E, 
ii)  P(*,B)  is  E-measurable  for  all  B  €  E. 

One  should  think  of  P(x,B)  as  representing  the  one-step  transition  probability 
of  the  chain  passing  from  x  into  B.  The  analogous  n-step  transition  prob¬ 
abilities  are  then  given  through  the  Chapman-Kolmogorov  equations,  namely 

p"'^^(x,B)  =  /  P’^(y,B)  P(x,dy) 

E 

where,  of  course,  P^(x,B)  =  6  (B)  (6  (B)  is  1  or  0  depending  on  whether 

or  not  X  €  B) .  Given  a  kernel  P  and  an  initial  probability  p.  on 
(E,E),  one  can  construct  a  measure  P  on  (2,F)  =  (E  x  E  x  • • • ,  E  x  E  x  • • • ) 
such  that 

V<=‘o  «  'o-  «„) 

=  /  M-Cdx.)  /  P(x„,  dx  )  ■  •  ■  /  P(x  ,  ,  dx  ) 

B,  °  B,  °  ^  B  " 

0  1  n 
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where  X^(aj)  =  and  (co^,  co^^,  ...)  is  a  typical  element  of  2.  The 
above  constructicn  yields  a  process  {X^  :  n  1}  that  is  endowed  with  the 
Markov  property 


Wi « 'i’‘o . V 


P  {X  ,  1  €  bIx  } 
|j.  n+1  n 


and  is  referred  to  as  the  GSSMC  arsociated  with  kernel  P  and  initial 
distribution  p.. 


EXAMPLE  3.1.  Let  E  =  {0,  1,  ...}  and  put  P(i,B)  p^ . >  where  P  is 

J  ij 

a  stochastic  matrix  on  E.  This  gives  the  classical  countable  state  Markov 
chain. 


example  3.2.  Let  (W^  :  n  2.  0)  be  the  waiting  time  process  of  a  GI/G/1 
queue.  Suppose  that  u  and  v  are  the  interarrival  and  service  r.v.'s 
respectively,  and  that  v-u  has  a  density  f  (x) .  Then  E  =  [0,»)  and 

X 

P(x,B)  =  /  f(y-x)dy  +  6  (B)  /  f (y)dy  . 

B  0 

EXAMPLE  3.3,  Consider  a  model  that  imitates  the  dynamics  of  a  lake.  Let  S^, 
and  X^  represent  the  volume  of  water  stored  in  the  lake,  the  inflow  of 
water,  and  outflow  of  water  respectively,  at  time  i.  Then,  the  mass-balance 
equation 


(3.4) 


S 


i-1 


4 
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holds.  If  one  now  assumes  that  output  increases  linearly  with  storage  through 
the  relation  =  aS^  (0  <  a  <  1),  then  (3.4)  takes  the  form 


*^i-l  ^  ^i 


where  p  =  1/ (1+a)  and  =  (aZ^)/(l+a).  Finally,  the  further  assumption 
that  is  i.i.d.  with 

P{e  €  B}  =  p6q(B)  +  (1-p)  /  f(y)dy 

B 

yields  a  Markov  chain  model  for  the  outflows  ^  “  [0»”)  and 


P(x,B)  =  p6o„(B)  +  (1-p)  /  f(y-px)dy  . 

B 

EXAMPLE  3.5.  The  embedded  jump  process  (S,C)  introduced  in  Section  2  is  a 
GSSMC  on  state  space  Z  and  with  kernel 

P((s,c),  B) 

=*p(s';  s,i*)  n  /  f(y;  s' ,i,s,i*)dy  H  6  ^(B  ) 
i  €Ns,  B^  j  eo^,  '"i  J 

where  B  is  that  subset  of  Z  corresponding  to  the  GSMP  entering  state  s' 

with  the  i'th  clock  of  s'  set  to  a  value  in  B.. 

1 

The  numbers  p(s';  s,i*)  govern  state  transitions  of  the  GSMP  and  represent 

the  probability  of  a  jump  to  s'  from  s,  given  that  clock  i*  initiated 

the  jump.  The  rest  of  the  kernel  governs  clock  readings.  Those  clocks 

i  €  N  ,  (new  clocks)  are  set  stochastically  according  to  a  density 
s 
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f(y;  s',i,s,i*),  whereas  those  clocks  j  €  0^,  (old  clocks)  are  inherited  from 

the  previous  state  and  so  must  be  set  deterministically  at  the  previous  value 

c*. 

J 

Note  that  by  setting  T  =  c,(i*(C,))  +  •••  +  c  (i*(C  ))  (c.(k)  is  the 

nil  n  n  j 

value  of  the  k'th  clock  at  the  j'th  jump  of  X^) .  We  can  retrieve 

{X^  :  t  >  0},  the  GSMP,  from  { (S  ,C  )  :  n  >  0}  via 
t  —  ’  n  n  — 

00 

\  Vi»  • 

k*0 

Let  us  turn  now  to  a  recurrence  condition  for  GSSMC's  first  formulated 
by  ATHREYA  and  NEY  (1978)  and  NUMMELIN  (1978).  We  say  that  {X^  :  n  ^  0} 
is  (A, B.X.'-P fk)  recurrent  if  there  exist  A,  B  6  E,  a  positive  number 
an  integer  k,  and  a  probability  tp  on  B,  such  that: 

n 

Ir 

ii)  P  (x,E)  2  ^(E)  for  each  x  in  A  and  measurable  subset  E  of  B. 

In  our  setting,  this  recurrence  notion  is  in  fact  equivalent  to  one  first 
proposed  by  HARRIS  (1956) . 

To  gain  some  appreciation  for  the  significance  of  these  conditions,  we 
observe  that  in  the  (A,B,X,(p,l)  case,  we  can  decompose  P  over  A  as 

P(x,E)  =  Xtp(E)  +  (1-X)  Q(x,E) 

where  Q(x,*)  is  a  probability  on  (E,E).  The  key  idea  of  Athreya,  Ney  and 
Nummelin  was  to  exploit  this  decomposition  via  the  following  embedding. 

Let  E'  =  E  X  {0,1}  and  E'  be  the  associated  product  a-field.  We 
extend  P  to  a  kernel  P'  on  (E',E')  by  setting 

7 


1 


(  (l-\)  PCx,E),  X  i  A 
P’ ((x,S),  E  X  {0})  =  { 

(  (1-X)  Q(x,E),  X  e  A 


(  XP(x,E) ,  X  )?  A 

P’((x.6),  E  X  {1})  =  j 

'■  Xtp(E  n  B),  X  e  A 


For  a  probability  p.  on  (E,E),  define  ix'  on  (E',E’)  by 


tx'(E  X  {i})  =  SqU)  (1-X)  h(E)  +  6j^(i)X  11(E) 


Then,  it  can  be  readily  verified  that  the  Markov  chain  X'  =  (X  ,  5  )  on 

n  n  n 

(E',E’)  associated  xd.th  P' ,  [i*  has  the  property  that  the  coordinate  process 
{Xjj  :  n  0}  is  the  original  GSSMC  on  (E,E)  associated  with  P,  ii. 

The  importance  of  this  embedding  is  that  it  furnishes  one  with  a 
sequence  of  regeneration  times  Tj^  defined  in  terms  of  X^  via 


T,  =  inf{n  >  1  :  X  ,  €  A,  6  •  1}  , 

1  —  n-1  n 

T,  =•  inf{n  >T,  ^rX  ,€A,  6  =1}  ,  k>l. 
k  k-1  n-1  n  ’ 


The  regenerative  character  of  X'  guarantees  "nice"  ergodic  behavior  for 

n 

X  .  For  example,  if  we  assume  that  E'(T--T, )  <  then  the  classical 
n  4  1 

regenerative  theory  shows  that  (e.g. ,  SMITH  (1955)) 


i  j  f<\)  -t- V«> 


a.s. 


where 
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'1 


T  -1 
2  ^ 


tt(B)  =  E'C  I  5^  (B)3  /E'(T2-T^) 


j=T,  J 


Estimation  of  r  is  a  common  goal  of  simulators.  The  above  discussion 
suggests  an  "extended  regenerative  method"  (ERM)  for  producing  confidence 
intervals  for  r  based  on  the  sequence. 

1.  Generate  the  sequence  { (X  ,  6  )  :  n  >  0}. 

n  n  — 

Tk+1 

2.  Let  =  I  and  form 

=  C  I  Y^/C  I 

k=l  k=l 

®11^"^  “  ^  \  “  n(n-l)  ^kP 

®12^''^  ”  \  \  "  n(n-l)  ^ 

•1  «  1  n  2 

s-,(n)  =  - r  ^  a, - 7 - :pr  (  1  a,") 

22  n-1  k  n(n-l)  ''  k' 


s^(n)  =  s^j^(n)  -  2r(n)  +  f^(n)  822(11)  . 


3.  To  fom  a  100(1-6)%  confidence  interval  for  r  choose  z  so  that 

6 

^(.z.)  =  1  -  6/2,  where  i  is  the  standard  normal  distribution  function, 
o 


Then 


.  z  s(n)  ^  z  s(n) 

r(n)  -  - ,  r(n)  +  * 


n 

^  \ 

k=l 

is  the  desired  confidence  interval. 


n 

I  \ 

k=l 
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The  above  steps  can  all  be  justified  under  the  assumption  that 
2 

0  <  ECY^  -  ra^)  <  Incidentally,  although  Step  1  appears  to  necessitate 

the  ability  to  generate  deviates  distributed  according  to  Q(x,').  this 
difficulty  can  be  avoided  by  using  an  acceptance-rejection  technique  ([8],  p.  57). 


EXAMPLE  3.1  (continued).  Suppose  that  X  =  {X^  :  n  _>  0}  is  an  irreducible, 
recurrent  Markov  chain  with  transition  matrix  P  =  Then,  X  is 

({0},  E,  1,  Pq  ,1)  recurrent  and  T^  =  Nj^+1,  where  is  the  k'th  hitting 

time  of  0.  The  EBM  reduces  here  to  the  classical  regenerative  technique 


(modulo  a  shift  to  the  right). 


EXAMPLE  3.2  (continued).  Assuming  that  Ev  ^  Eu,  {W^  :  n  0}  is  ({0},  E,  1, 
P(0,-),  1)  recurrent,  and  again  T^  =  Nj^+1,  when  is  the  k'th  hitting 

time  of  0,  reducing  our  estimation  technique  to  the  usual  one  (again, 
modulo  a  shift). 


EXAMPLE  3.3  (continued).  Assuming  that  Es^  <  and  that  f  is  positive 

and  continuous  over  [0,  +«),  {X  ;  n  >  0}  is  (I0,b],  E,  X,  (p ,  1)  recurrent 

n  — 

where 

'Ky)  =  (1-p)  min  f(y-px) 

0  £  X  <  b 

oo 

X  =  /  t(y)dy;  ((>(y)  =  i(y)/X  . 

0 

Here  the  T  's  form  a  subsequence  of  the  hitting  times  of  [0,b]  —  this 

eC 

reflects  the  fact  that  X^  returns  to  no  point  infinitely  often.  The  ERM 
is  applicable  here,  whereas  the  classical  regenerative  method  is  not. 


i 


EXAMPLE  3,5  (continued).  We  say  that  the  GSMP  has  a  single  set  if  there 
exists  a  state  s'  with  which  is  associated  only  one  clock.  Then,  given 
that  the  GSMP  hits  {s'}  infinitely  often,  (S,C)  is  (A,  Z,  1,  PCCs',^,  •),  1) 
recurrent  where  A  =  {(s,c)  €  Z  :  s  =  s'}.  Here,  where  the  N^^'s 

are  consecutive  hitting  times  of  A. 

It  should  be  remarked  that  in  the  context  of  a  GSMP,  a  simulator  is 
often  interested  in  the  continuous-time  quantity 

t 

r"  =  11m  /  f(X  )ds/t 

t  -►  oo  0  ® 


This  can  be  estimated  by  the  ERM,  provided  that  the  definitions  of  and 

a,  are  modified  to 
k 


£<S^)  Cj<l.(C^)> 


[  c  (i*(C  )) 


The  above  discussion  focussed  on  (A,  B,  X,  (p ,  1)  GSSMC's.  A  fundamental 
difficulty  arises  in  the  (A,  B,  X,  cp,  k)  case,  a  difficulty  that  can  be 
partially  circumvented  via  an  embedding  that  endows  the  chain  with  an 
environment  that  is  "loosely  regenerative"  in  the  sense  of  SMITH  (1955).  An 
estimation  procedure  can  then  be  developed  that  bears  close  resanblance 
to  the  "extended  regenerative  method"  outlined  above  (see  [8]). 

Before  leaving  this  topic,  it  should  be  mentioned  that  a  number  of 
difficulties  remain  in  terms  of  implanentation  of  the  ERM.  The  most  fundamental 
problem  is  that  the  decomposition  of  P  over  A  requires  an  explicit  form 
for  the  kernel  over  that  set.  The  "event-scheduling"  approach  normally  used 
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by  simulators  to  generate  sample  paths  does  not  require  an  explicit 
representation  for  the  kernel,  and  so  a  simulator  is  left  with  the  burden¬ 
some  task  of  calculating  such  a  form.  Additional  difficulties  arise  in 
determining  appropriate  X,  <p,  and  A,  although  here  simple  numerical 
techniques  would  be  applicable  ([8],  p.  52). 


4.  Random  Blocking  Method  for  GSSMC’s 

Our  major  Incentive  in  studying  GSSMC's  here  has  been  in  terms  of 
application  to  GSMP's.  It  turns  out  that  GSMP's  possessing  no  single  set 
cannot  be  (A,  B,  X,  (p,  1)  recurrent  ([8],  p.  60),  and  hence  the  ERM 
discussed  in  the  previous  section  does  not  apply.  This  factor  together 
with  the  ERM-related  difficulties  already  mentioned,  motivates  development 
of  other  methods. 

Suppose  that  {X^  :  n  2  0}  is  an  (A,  B,  X,  (p,  k)  GSSMC  with  invariant 
probability  n .  This  will  in  fact  be  the  case  for  the  embedded  jump  process 
CS,C),  corresponding  to  a  GSMP  {X^  :  t  ^  0},  provided  that  (see  [9]) 

i)  the  state  space  S  is  finite 

ii)  the  "road  map"  p(s',  s,  i*)  satisfies  a  natural  irreducibillty 
condition  (see  [7],  p.  16) 

iii)  the  densities  f(';  s',  i,  s,  i*)  are  positive  and  continuous 
on  [0,°°),  with  finite  mean. 

For  such  an  (A,  B,  X,  cp,  k)  recurrent  {X  :  n  >  0}  one  can  show  that 

n  — 

if  E„If  (X)  I  <  ■+«,  then 

^  Jo 
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a.  s. 


for  any  M-  on  (E,E).  A  simulator  commonly  wishes  to  obtain  an  estimate 
for  r,  together  with  an  associated  confidence  interval. 

Let  T2»  be  the  consecutive  hitting  times  of  the  set  A.  The 

"random  blocking  method"  (RBM)  hinges  on  the  observation,  due  essentially  to 
OREY  (1959),  that 


in  »  I  o  *  •  •  •  »  ) 

k  ^k  k+1 


is  a  Doeblin  recurrent  Markov  chain  with  a  single  ergodic  set  (see  DOOB 
(1953)  for  definitions  and  results).  Under  the  assumption  that  is 

aperiodic  (this  will  generally  be  the  case  for  GSMP's),  functions  of 
will  enjoy  a  central  limit  theorem  with  a  variance  constant  of  the  form 


(4.1) 


cr^  =  a^(Zn)  +  2  J  cov(Zq,  Z^^) 


k=l 


By  truncating  the  infinite  sura  and  estimating  the  finite  number  of  remaining 
terms  in  the  series  by  the  standard  sample  moments,  we  obtain  the  RBM. 

1.  Choose  a  truncation  number  m  (the  number  of  covariance  terms  of 

(4.1)  to  be  retained). 

T  -1 

2.  Put  “k  =  Vl  -  \ 


f(n)  =  C  I  O/C  I  OjjP 

k=l  k-1 

n-^ 


“  n-l  ^k  n-e^  \  \+l 


r(n) 


n-i 


*2,  ..  n-^ 

+  — ^  I  \  “i, 


n-e  \  \+Z  '  n-l  "k  '"k+« 

k=l  k=l 
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/  E  {  I  I  f(X  )|^'^}lT(dx)  <  -H»  , 

A  ^  j=l  J 

for  some  5  >  0  ([9]).  The  GSMP's,  the  RBM  can  be  modified  in  the  same  way 

as  the  ERM  so  as  to  provide  confidence  intervals  for  continuous-time  quantities 
of  the  form  (3.5). 

Observe  that  when  m  =  0,  this  technique  reduces  to  the  "approximate 
regenerative  method"  (ARM)  of  CRANE  and  IGLEHART  (1975).  Hence  one  can  think 
of  the  RBM  as  a  second-order  refinement  of  the  ARM.  It  should  be  noted, 
however,  that  the  RBM  involves  the  undesirable  element  of  having  to  make  an 
a  priori  judgment  as  to  an  appropriate  value  of  the  truncation  index  m. 

This  is  a  problem  to  which  we  intend  to  devote  more  attention. 
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Multivariate  Autoregressive  Method 

The  last  method  we  shall  discuss  is  the  multivariate  autoregressive 

method  (MARM) .  Let  {X  :  n  >  0}  be  a  vector-valued  strictly  stationary 

~n  — 

process  with  mean  vector  and  covariance  function  ^(h) 

=*  E{[X  -u]  [X  Then  under  some  regularity  conditions  (see  BILLINGSLEY 

~n  ~n+h 

(1968),  Theorem  20.1)  the  following  central  limit  theorem  holds  as  n  -*■  <*>: 


N(0,Z)  , 


where  Z  =  Z(0)  +27“  Z(h). 

To  apply  this  result  to  simulation  output  analysis  we  need  a  method 
to  estimate  the  covariance  matrix  Z. 

Let  f (X)  be  the  matrix-valued  spectral  density  function  for  the 
process  {X  :  n  _>  namely. 


f  (X) 


I  Z(h) 


X  real 


Observe  that  Z  *  f (0) .  The  function  f  can  be  approximated  arbitrarily 
closely  by  the  corresponding  spectral  density  of  a  multivariate  autoregressive 
(MAR)  process  fitted  to  the  process  {X  :  n  >  0}.  To  fit  a  MAR  process  of 
order  p  we  must  find  matrices  {i^  :  0  £  k  ^  p}  so  that 

>  A.  X  .  =  Z 
rjs.  .v.n-k 


where  the  Z  's  are  i.i.d.  normal  with  mean  0  and  known  convariance  Ji.  We  select 
''-n  ~ 

the  order  p  of  the  MAR  process  by  applying  the  AlC-criterion  of  AKAIKE  (1976), 

(19  ).  Once  the  model  is  fitted,  it  is  a  simple  computation  to  find  the 
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spectral  density  function  of  the  MAR  process.  This  function  evaluated  at 
zero  then  provides  our  estimate  for  Z. 

This  method  and  related  ones  will  be  developed  in  detail  in  JOW  (1981). 
In  Section  6  data  will  be  presented  from  an  application  of  the  method  to  the 
lake  model  presented  in  Section  3. 


6.  Application  to  the  Lake  Model 

In  this  section  we  illustrate  the  application  of  the  three  methods  to 
the  lake  model  presented  in  EXAMPLE  2.  .  Recall  the  model  is  generated  by 
the  recursion 


X  =  PX„  ,  +  e  ,  n  >  0  , 

n  n-1  n  — 

where  the  e  's  are  i.i.d.  and  5  <  1.  It  can-  be  shown  that  X  =>  X  as 
n  n 

n  oo  with  E{X}  =  EfZ^^}  =  (l-o)  ^  E{s^}.  For  this  simulation  we  have  taken 

the  e  's  to  nave  the  distribution  Pic  <  x}  =  1  -  (l-o)e  x  >  0,  which 
n  n  —  — 

results  in  X  being  exponential  with  parameter  1.  The  simulations  were 

carried  out  to  estimate  E{X}.  For  the  ERM  the  return  set  A  =  [0,b] , 

where  b  =  (-log  p)/(l-p).  This  value  can  be  shown  to  maximize  the  expected 

number  of  regenerations.  The  value  o  =  0.75  was  used  which  makes  b  =  1.15. 

For  the  RBM  the  return  set  A  =  [.75,  1.25].  The  infinite  sum  of  covariance 

terms  in  the  RBM  was  truncated  at  m  =  0,  5,  10,  25).  For  the  MARM  the 

2 

vector  of  observations  used  was  (X  ,  X  ,  !.«  ,  ti  (X  )).  A  total  of  10,000 

n  n  [U,  in  /J  n 

observations  were  generated  in  each  of  30  replications. 
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TABLE  6.1.  Simulation  Results  for  Lake  Model 
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